Click any tag below to further narrow down your results
Links
Sean Goedecke argues that LLMs don't have fixed goals and their internal workings can't yield the kind of motive-level understanding needed to guarantee safety, so "solving alignment" for them is incoherent. He concludes that mechanistic interpretability, though useful, can't deliver the confidence required to prevent rogue superintelligent behavior.
- LLMs are trained to imitate human text, so they lack a single fixed goal that could be formally specified, unlike chess engines or the classic "blind optimizer" paperclip model.
- Mechanistic interpretability can trace calculations but not motive, and it can only label behaviors we've already elicited, leaving many failure modes unobserved.
- Feature labels are guesses that can't be distinguished from alternatives (the "kindness unless 2030" problem), and the combinatorial complexity per token is too large for humans to check.
- Current mitigations like chain-of-thought monitoring and sandboxing are useful but judged insufficient against a superintelligence.
+ alignment
+ llm
+ mechanistic interpretability
sean goedecke
+ superintelligence
+ ai safety
kripke